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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including tine fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
03/16/2009 has been entered. 

Response to Arguments 

2. Applicants arguments with respect to claims 1 -3, 9-1 2, 1 5-1 9, 21 , 23, 25-30, 32- 
39, 41-43, 46-47 have been considered but are moot in view of the new grounds of 
rejection. Examiner has incorporated Scarano et al. US 20040083099 A1 (hereinafter 
Scarano) in view of Petrushin US 20020194002 Al (hereinafter Petrushin) to address 
audio interaction in a call center, wherein an area of interest within an audio signal is 
identified. 

Claim Objections 

3. Claim 15 objected to because of the following informalities: Claim 15 depends 
upon cancelled claim 14. Appropriate correction is required. 
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4. Claim 36 objected to because of the following informalities: Claim 36 contains 
typographical errors, wherein "dual tome mulfi frequency " is construed to be "dual tone 
multi frequency" in light of the corresponding method claim 19. Appropriate correction is 
required. 



Specification 

5. The specification is objected to as failing to provide proper antecedent basis for 
the claimed subject matter. See 37 CFR 1 .75(d)(1 ) and MPEP § 608.01 (o). Correction 
of the following is required: Claim 47 discloses "the reducing step is repeated two or 
more times" with no description within the disclosure. 



Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 1 22(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 
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7. Claims 1-3, 9, 10, 15-17, 19, 21, 23, 25-27, 29, 32-39, 41, and 46-47 are rejected 
under 35 U.S.C. 102(e) as being anticipated by Scarano et al. US 20040083099 A1 
(hereinafter Scarano). 

Re claims 1 and 19, Scarano teaches an apparatus for event-driven content 
analysis of an audio captured interaction captured in a call center, within a 
computerized system having a processing unit and a storage unit ([0161]), the 
apparatus comprising the elements of: 

an audio or video recording device for recording the audio interaction ([0162]); 

a pivot spot defining component for automatically marking an at least one time 
position in the audio interaction that indicates the occurrence of an at least one pre- 
defined event or data item ([0030]); 

a first analysis component (Fig. 19 item 1902); 

a region of interest defining component for defining an initial region of interest, by 
determining the time limits of an at least one segment of the audio interaction ([0108]), 
the segment containing the time position of a pivot spot ([01 15]), 

and for activating the first analysis component for dynamically reducing the time 
limits of the initial region of interest to obtain the region of interest ([0098-0102], phrase 
isolation); 

a second analysis component for analyzing the region of interest of the audio 
interaction ([0098-0102] & Fig. 19 items 1906-1908), 
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wherein tlie second analysis component requires more computing resources 
than the first analysis component (Fig. 19, when later stages 1906-1908 are reached, 
more resources are inherently used). 

Re claim 2, Scarano teaches the apparatus of claim 1 further comprising a 
content analysis input selector component to determine an at least one input or 
parameter for an at least one analyzer component ([01 15]) 

Re claim 3, Scarano teaches the apparatus of claim 1 further comprises an 
analysis type selector component to identify and to select an at least one analyzer 
component type for determining the Region of Interest ([0098-0102]. 

Re claim 9, Scarano teaches the apparatus of claim 1 further comprises the 
element of an audio analyzer component to analyze the audio elements of the 
interaction data ([0108]) 

Re claim 10, Scarano teaches the apparatus of claim 1 wherein the first analysis 
component of the second analysis component is a computer telephony interface events 
analyzer component for analyzing at least one common telephony events associated 
with the interaction data ([0006]). 
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Re claim 15, Scarano teaches tine apparatus of claim 14 wherein the interaction 
media is at least one data packet carrying voice or other media over internet protocol 
([0156]). 

Re claim 16, Scarano teaches the apparatus of claim 1 wherein the region of 
interest is a specific segment of the interaction media that is analyzed to extract 
meaningful interaction-specific information in an organization ([0108]) 

Re claim 17, Scarano teaches the apparatus of claim 1 wherein the interaction 
associated with an at least one computer telephony integration event occurring during 
the interaction ([0006]). 

Re claim 21 , Scarano teaches the method of claim 19 further comprising the step 
of selecting a method for the analysis of the at least one interaction media based on the 
at least one event associated with the interaction ([0108]). 

Re claim 23, Scarano teaches the method of claim 19 further comprising the step 
of selecting the parameters to be used in the at least one analysis instruction step on 
the at least one segment of the interaction ([0098-0102] & Fig. 19 items 1906-1908). 

Re claim 25, Scarano teaches the method of claim 19 wherein the region of 
interest is predetermined by a user or an apparatus ([01 15]) 
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Re claim 26, Scarano teaclies tine metliod of claim 19 further comprises the 
steps of receiving interaction data and associated meta-data from an at least one 
interaction ([0078]). 

Re claim 27, Scarano teaches the method of claim 19 wherein the at least one 
first analysis or the second analysis instruction step comprises the step of analyzing 
the-speech elements of the interaction data for the presence of pre-defined words or 
phrases ([0108]). 

Re claim 29, Scarano teaches the method of claim 19 wherein the at least one 
first analysis instruction step or the second analysis comprises the steps of analyzing 
t4ae-speech elements of the interaction data for pre-defined speech patterns ([0108]). 

Re claim 32, Scarano teaches the method of claim 19 further comprises 
performing an at least one content analysis step during capturing of the interaction data 
and the interaction meta-data ([0065], start and end times embedded). 

Re claim 33, Scarano teaches the method of claim 19 wherein the at least one 
pivot spot or the region of interest ([01 15]) are determined based on an event external 
to the interaction ([0055], separate CTI actions). 
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Re claims 34 and 35, Scarano teaches the apparatus of claim 1 wherein the pivot 
spot is determined using at least one item selected from the group consisting of: a 
Computer Telephony Integration event ([0065]); a screen event; an emotional level; and 
a spotted word. 

Re claims 36 and 37, Scarano teaches the apparatus of claim 1 wherein the first 
analysis component used for reducing the initial region of interest is selected form the 
group consisting of: a predetermined length, speaker separation component, audio 
analysis component, event analysis component, a Computer Telephony Integration 
event analysis component ([0065]), CRM event analysis component, a screen event 
analysis component; an emotional level analysis component, a word spotting analysis 
component, audio event analysis component, dual tone multi frequency (DTMF) event 
analysis component, and even priority analysis component. 

Re claims 38 and 39, Scarano teaches the apparatus of claim 1 wherein the 
captured interaction is between an agent and a customer ([0028], well known call center 
operations between caller/customer and agent). 

Re claim 41 , Scarano teaches the method of claim 19 wherein the method is 
used for detecting customer churn indications ([0098-0102], cancel service), wherein 
the pivot spot is defined using a CTI hold event or a cancellation-related screen event; 
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and wherein tlie region of interest is defined using emotion analysis or word spotting 
([0065], audio segments). 

Re claim 46, Scarano teaches the apparatus of claim 1 wherein the at least one 
pivot spot or the region of interest are determined based on an event external to the 
interaction ([0055], separate CTI actions). 

Re claim 47, Scarano teaches the method of claim 19 wherein the reducing step 
is repeated two or more times ([0036], more than one audio segment from multiple 
extraction operations). 



Claim Rejections - 35 USC § 103 
8. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



9. Claims 1 1 , 1 8, and 30 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Scarano et al. US 20040083099 A1 (hereinafter Scarano) in view of 
Bscheider et al. US 6937706 B2 (hereinafter Bscheider). 
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Re claim 1 1 , Scarano fails to teach the apparatus of claim 1 further comprises a 
screen event analyzer component to identify and capture an at least one screen and an 
at least one screen event associated with the interaction data and capture at least one 
screen based on the screen event 

Bscheider teaches that the term "Call Control" refers to the part of the metadata 
concerning the creation and termination of call records. The term "Media" refers to the 
actual data that is being recorded. This term is used interchangeably with audio since 
the primary design of the CRG is to support audio recording. However, the CRG could 
apply to any data being recorded including multimedia or screen image data (Col. 31 
lines 29-40). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Scarano to incorporate a screen event 
analyzer component to identify and capture an at least one screen and an at least one 
screen event associated with the interaction data and capture at least one screen based 
on the screen event as taught by Bscheider to allow for the integration of image/screen 
capture in accordance with synchronized audio, wherein metadata is present that 
indicates audio and image association data (Col. 31 lines 29-40). 

Re claims 18, Scarano fails to teach the apparatus of claim 1 wherein the 
interaction meta-data is associated with the at least one screen event. 

Bscheider teaches that the term "Call Control" refers to the part of the metadata 
concerning the creation and termination of call records. The term "Media" refers to the 
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actual data that is being recorded. This term is used interchangeably with audio since 
the primary design of the CRG is to support audio recording. However, the CRG could 
apply to any data being recorded including multimedia or screen image data (Col. 31 
lines 29-40). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Scarano to incorporate meta-data is 
associated with the at least one screen event as taught by Bscheider to allow for the 
integration of image/screen capture in accordance with synchronized audio, wherein 
metadata is present that indicates audio and image association data (Col. 31 lines 29- 
40). 

Re claim 30, Scarano teaches the method of claim 19 further comprises the 
steps of: 

identifying an at least one pre-defined computer telephony integrated event in the 
interaction data ([0108]); 

However, Scarano fails to teach identifying an at least one pre-defined screen 
event in the interaction data 

Bscheider teaches that the term "Call Control" refers to the part of the metadata 
concerning the creation and termination of call records. The term "Media" refers to the 
actual data that is being recorded. This term is used interchangeably with audio since 
the primary design of the CRG is to support audio recording. However, the CRG could 
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apply to any data being recorded including multimedia or screen image data (Col. 31 
lines 29-40). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Scarano to incorporate identifying an at 
least one pre-defined screen event in the interaction data as taught by Bscheider to 
allow for the integration of image/screen capture in accordance with synchronized 
audio, wherein metadata is present that indicates audio and image association data 
(Col. 31 lines 29-40). 

10. Claims 12 and 28 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Scarano et al. US 20040083099 A1 (hereinafter Scarano) in view of Petrushin 
US 20020194002 Al (hereinafter Petrushin). 

Re claims 12, Scarano teaches the apparatus of claim 9, wherein the audio 
analyzer component further comprises the elements of: 

a word spotting component to locate and identify pre-defined terms or patterns in 
the speech elements of the interaction data ([01 15]); 

a talk analyzer component to identify and locate specific pre-defined speech 
events in the speech elements of the information data ([0108]) 

However, Scarano fails to teach an emotion analysis component to locate and 
identify positive or negative emotions in the interaction data 
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Petrushin teaches logic for receiving and analyzing a speech signal, logic for 
dividing the speech signal, and logic for extracting at least one feature from the speech 
signal. The system comprises logic for calculating statistics of the speech, and logic for 
at least one neural network for classifying the speech as belonging to at least one of a 
finite number of emotional states. The system also comprises logic for outputting an 
indication of the at least one emotional state (Petrushin [001 1]). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Scarano to incorporate an emotion 
analysis component to locate and identify positive or negative emotions in the 
interaction data as taught by Petrushin to allow for the classification of a speech based 
on emotion, wherein features from a signal are used to determine an emotional state 
(Petrushin [0011]). 

Re claim 28, Scarano teaches the method of claim 19 wherein the at least one 
first analysis instruction step or the second analysis comprises the step of analyzing 
the-speech elements of the interaction data ([01 15]) to detect positive and negative 
emotions. 

However, Scarano fails to teach analyzing the-speech elements of the interaction 
data ([01 1 5]) to detect positive and negative emotions. 

Petrushin teaches logic for receiving and analyzing a speech signal, logic for 
dividing the speech signal, and logic for extracting at least one feature from the speech 
signal. The system comprises logic for calculating statistics of the speech, and logic for 
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at least one neural network for classifying the speech as belonging to at least one of a 
finite number of emotional states. The system also comprises logic for outputting an 
indication of the at least one emotional state (Petrushin [001 1]). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Scarano to incorporate analyzing the- 
speech elements of the interaction data ([01 15]) to detect positive and negative 
emotions as taught by Petrushin to allow for the classification of a speech based on 
emotion, wherein features from a signal are used to determine an emotional state 
(Petrushin [0011]). 

11. Claim 42 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Scarano et al. US 20040083099 A1 (hereinafter Scarano) in view of Eilbacher et al. 
US 6724887 B1 (hereinafter Eilbacher). 

Re claim 42, Scarano fails to teach the method of claim 19 wherein the method is 
used for verifying that an agent requested a customer's permission to put the customer 
on hold, wherein the pivot spot is the time the agent put the customer on hold, the initial 
region of interest is the whole interaction, and wherein the region of interest is defined 
by a first predetermined number of seconds prior to the pivot spot and a second 
predetermined number of seconds following the hold. 

Eilbacher teaches a contact center 200 of FIG. 2, and in particular a telephone 
call center. Referring to FIG. 3, customers 100 access the contact center through the 
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public switclied teleplione network (PSTN) 101 and an automatic call distribution 
system 1 02 (PBX/ACD) directs the communication to one of a plurality of agent work 
stations 104. Each agent work station 104 includes, for example, a computer and a 
telephone set. Communications are directed to the agent stations 104 based on the 
availability of the agent. In those contact centers handling communications for a 
number of different clients, communications to a particular client may be routed to a 
finite group of agents specifically trained to respond to the needs of that customer or 
that client. Alternatively, the PBX/ACD 102 may include an interactive voice response 
(IVR) system that presents an audio menu to a customer, requesting a response by way 
of the customer's telephone key pad or by way of a voice response. Then, a call is 
directed to a particular group of agent stations 104 or to a particular information retrieval 
system, based on the responses of the customer. For example, the system can provide 
the customer 100 with the address to which products should be returned or the Internet 
address for obtaining additional product information. All data associated with the 
customer's communication and the agents responsive interaction with the customer may 
be recorded by a monitor module 210 within monitoring system 204. Examples of the 
data typically recorded by a telephone call center system include the audio 
communication between the customer and the agent, key pad data input by the 
customer, screens viewed by the agent on the computer at the agent station 1 04 
(carried by data line 105), the start and end time for the customer's communication, the 
identity of the customer, including the originating telephone number and the called 
number, the identity of the various agents servicing the communications, the length of 
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time the customer is on hold and the steps the customer navigated before terminating 
the communication (Eilbacher Col. 8 lines 29-67). 

Further, Eilbacher teaches incoming and outgoing calls can be recorded in their 
entirety; particular calls can be identified for recording, such as by client or agent; and 
calls can be recorded by event, such as calls exceeding five minutes. If "cradle-to- 
grave" recording is used, then all information related to a particular telephone call or 
caller-initiated transaction is recorded, from the time the call enters the contact center to 
the later of: the caller hanging up or the agent completing the transaction. All of the 
interactions during the call are recorded, including interaction with the IVR system, time 
spent on hold, data keyed through the caller's key pad, conversations with the agent, 
and screens displayed by the agent at his/her station 104 during the transaction. These 
types of recordings allow for evaluation of the full customer experience throughout the 
transaction. As an example, the length of time a customer was on hold during a 
purchase transaction can be analyzed as a possible deterrent to completing a purchase. 
Such information may be used by contact center managers to modify their procedures, 
staffing, and/or equipment to improve the customer's experience when using the contact 
center. The comprehensiveness of the data capture of the present invention also allows 
for the subsequent verification of transaction content. For example, a dispute over what 
information was verbally provided by a caller applying for insurance coverage over the 
telephone can easily be resolved by replaying the application call in its entirety. 
Whether a customer selected size 10 can also be proven, as can whether the 
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customer/investor authorized the purchase of 100 shares of a particular stock. 
(Eilbacher Col. 9 lines 10-39). 

Furthermore, Eilbacher teaches types of parameters which can be analyzed by 
the customer experience analyzing unit 208 include the number of key strokes entered 
by the customer during a telephone call, the length of a telephone call, time on hold, 
number of transfers, or length of a queue. That is, if the length of the telephone call, the 
number of key strokes entered during the call or the length of a queue exceeded 
predetermined levels, the customer experience analyzing unit 208 can determine that 
the communication was likely unsatisfactory. In addition, speech detection or word 
spotting can be used to detect certain inflammatory words such as curse words. For 
example, in the case of word spotting, an analysis is performed on recorded audio such 
as a telephone call. The audio is automatically processed, searching for any key words 
on a predefined list which have been identified as cause for concern. If any of the 
words are found, the call is marked as a potentially negative customer experience. This 
word spotting analysis can be done separately, or in addition to the stress analysis. 
Similarly, in connection with an e-mail communication, a text search can be used to look 
for words such as curse words, which might tend to indicate an unsatisfactory customer 
experience (Eilbacher Col. 11 lines 25-61). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Scarano to incorporate verifying that an 
agent requested a customer's permission to put the customer on hold, wherein the pivot 
spot is the time the agent put the customer on hold, the initial region of interest is the 
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whole interaction, and wherein the region of interest is defined by a first predetermined 
number of seconds prior to the pivot spot and a second predetermined number of 
seconds following the hold as taught by Eilbacher to allow for the monitoring of a 
customer and agent interaction, wherein various retrieval methods are implemented 
based on responses (Eilbacher Col. 8 lines 29-67) and key locations of media selected 
based on the customer responses (i.e. images, keywords, etc.) to create and 
ideal/optimal retrieval of information (Eilbacher Col. 11 lines 25-61). 



12. Claim 43 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Scarano et al. US 20040083099 A1 (hereinafter Scarano) in view of Eilbacher et al. 
US 6724887 B1 (hereinafter Eilbacher) and further in view of Bernard et al. US 
5918213 A (hereinafter Bernard). 

Re claim 43, Scarano in view of Eilbacher fails to teach the method of claim 1 9 
wherein the method is used for measuring the effectiveness of a promotion offer to a 
customer requesting the termination of the service, wherein the pivot spot is the time of 
a screen event related to offering a promotion or to an account being saved or lost, and 
wherein the region of interest is defined by a first predetermined number of seconds 
prior to the pivot spot. 

Bernard teaches that promotional items are offered to a customer 1 82 based on 
his or her calling and purchasing history. For example, in one embodiment, the 
automated product purchasing system review calling and purchasing statistics 
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maintained for a shopper 182. Statistics can be maintained by VRU 104, interactive 
transaction database 1 12, or even by reporting database 438. If these statistics indicate 
that the shopper is a particularly good customer of the automated product purchasing 
system, interface unit 104 may offer a promotional or special item to that shopper 182. 
For example, where shopper 182 is a frequent purchaser, interface unit 104 may inform 
him or her that upon the next purchase, he or she will receive a bonus CD (Bernard Col. 
51 lines 29-41). 

Further, Bernard teaches a caller 182 decides to delete all of the items in his or 
her virtual shopping cart, as indicated by input step 3724, VRU 104 simply deletes these 
items from the order. This deletion step is illustrated by a step 3728. In one 
embodiment this is accomplished by deleting all the order information from the previous 
call and beginning anew with fresh order information for the present call. In an 
alternative embodiment, the deletion is accomplished by simply removing the items' 
catalog ID numbers 1008 from the order information in interactive transaction database 
1 12. A confirmation script can be played by VRU 104 announcing that the order has 
been canceled. As with the other options, at this time the caller is returned to the 
shopping mode where he or she can sample additional selections or terminate the 
phone call. Although not illustrated, in one embodiment, caller 182 is given the option of 
hearing a listing of the items in his or her virtual shopping cart before deciding whether 
to keep or cancel the order entirely. Finally, caller 182 may decide to individually review 
the items in the virtual shopping cart and determine whether each individual item is to 
be kept. If this is the case, caller 182 elects to review the items on hold as illustrated by 
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input step 3732. In response, in a step 3736, VRU 104 reviews the order with caller 
182. In one embodiment, this is accomplished by a process similar to that illustrated in 
FIG. 36 where each item is reviewed one at a time, and caller 182 selects whether to 
accept or delete each item. Once the review order process is complete, caller 182 is 
forwarded to the shopping mode where he or she can sample additional selections, 
Immediately purchase the selections remaining in his or her virtual shopping cart, or 
terminate the call (Bernard Col. 50 lines 1-32). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the Invention to modify the system of Scarano in view of Ellbacher to Incorporate 
measuring the effectiveness of a promotion offer to a customer requesting the 
termination of the service, wherein the pivot spot is the time of a screen event related to 
offering a promotion or to an account being saved or lost, and wherein the region of 
Interest Is defined by a first predetermined number of seconds prior to the pivot spot as 
taught by Bernard to allow for the automated and statistical determination of whether a 
promotional/coupon should be applied to a caller based on the callers history (Bernard 
Col. 51 lines 29-41), wherein an order can be cancelled based on a caller history 
(Bernard Col. 50 lines 1-32). 



Conclusion 

1 3. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. US 6859776 B1, US 20040062364 A1, US 6404857 B1, US 
6879685 B1 , US 6898277 B1 , US 7003079 B1 , US 7058376 B2, US 20050030374 A1 . 
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Any inquiry concerning tliis communication or earlier communications from tine 
examiner should be directed to Michael C. Colucci whose telephone number is (571)- 
270-1847. The examiner can normally be reached on 9:30 am - 6:00 pm, Monday- 
Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (571 )-272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Michael C Colucci/ 
Examiner, Art Unit 2626 
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/Ricliemond Dorvil/ 

Supervisory Patent Examiner, Art Unit 2626 



